cuda profiling

Overview of monitoring and optimizing GPUs for AI workloads: key metrics like utilization, memory, SM occupancy, throughput, and temperature; diagnosing bottlenecks from I/O stalls to inefficient kernels. Covers tools including nvidia-smi, DCGM, Nsight, PyTorch/TensorFlow profilers, Prometheus and Grafana, with dashboards, alerts, and workflows to profile, right-size, and reduce cost for training and inference at scale.